Apparatus and method for converting between different video formats

ABSTRACT

A method and apparatus for converting between different video formats, which provide smooth video motion by eliminating unnatural effects which could be introduced in the process of video format conversion. A frame interpolator produces interpolated frames from a first video signal given in a first video format by using motion vectors of the first video signal. From the interpolated frames, a video signal generator produces a second video signal in a second video format that is incompatible with the first video format.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an apparatus and method for convertingbetween different video formats, and more particularly, to an apparatusand method for converting between different video formats to enablevisual communication between remote users of different televisionsystems.

2. Description of the Related Art

There are three major standard television systems available today: NTSC,PAL, and SECAM. NTSC is used in the United States and Japan, while PALand SECAM mainly in European nations. For comparison, FIG. 18 shows somekey specifications of those three different video standards, including:frame frequency, the number of effective picture elements (pels orpixels) per line, and the number of effective lines per frame. AlthoughPAL and SECAM seem to share some common parameters, SECAM is actuallyincompatible in operation with PAL.

As noted above, different video formats are used in different groups ofnations, and to solve this incompatibility problem in internationalvisual communications, it is necessary to introduce some standardizedvideo coding formats. Typical video formats for such purposes are H.261and H.263, recommendations from the International TelecommunicationsUnion-Telecommunications Standards Sector (ITU-T). They provide twocommon video formats called “Common Intermediate Format (CIF)” and“Quarter Common Intermediate Format (QCIF).” Some of their keyspecifications are shown in FIG. 19 to make a comparison in terms of thenumber of picture elements per line, the number of lines per frame, andframe frequency. In this table of FIG. 19, the symbol “Y” representsluminance signals, and “Cr” and “Cb” denote color difference signals.

FIG. 20 shows a simplified system structure for international digitalvideo communications between two countries using different televisionsystem standards. This video communications system interconnects an NTSCsystem 2 and a PAL system 4 by using CIF as a vehicle for relayingdigital video signals via a communications satellite 3. To convert thevideo format, a video signal processor 200 a is disposed at the NTSCside, and another video signal processor 200 b at the PAL side. Theformer processor 200 a comprises a format converter 201 a and a codec(coder/decoder) 202 a, while the latter processor 200 b comprises aformat converter 201 b and a codec 202 b. Such video signal processors200 a and 200 b are implemented as part of videoconferencing stations orvideophone terminals, for example.

Consider that the NTSC system 2 attempts to send video information tothe PAL system 4. The format converter 201 a in the video signalprocessor 200 a converts incoming NTSC video signal into a CIF videostream, which conforms to the common format for communications. This CIFvideo stream is then encoded by the codec 202 a and transmitted to thecommunications satellite 3 via a radio transmitter (not shown). Theradio signal retransmitted by the communications satellite 3 reaches areceiver (not shown) attached to the video signal processor 200 b. Thevideo signal, now in the form of an electrical signal, is supplied tothe codec 202 b for video decoding, where the original CIF video streamis reconstructed. The format converter 201 b converts this CIF videostream to a PAL video signal for use in the PAL system 4. It would beunderstood that video signal transmission in the opposite direction(i.e., from PAL to NTSC) can be achieved in a similar fashion.

As described above, conventional visual communications systems use theCIF as a common format to transport video information to remote sites.That is, the sender encodes video signals after converting from itslocal format to the CIF format, and the receiver converts from the CIFformat to its local format after decoding reception signals. In suchinternational video communications, conventional systems resolve thedifference in frame frequencies by subsampling or duplicating videoframes. More specifically, when a video signal having a certain framefrequency has to be converted to a lower frequency (e.g., 30 Hz to 25Hz), the source frames are subsampled, or decimated, at predeterminedintervals. On the other hand, when a video signal has to be converted toa higher frequency, some frames are sent twice to adjust the framefrequency.

However, the above-described video conversion techniques introduce someunnatural effects into objects' motion in a video, because of thediscontinuity of frame pictures. Viewers may perceive the drop orduplication of frames as awkward motion of objects, particularly whenthe video contains rapid motion such as a soccer ball flying in aparabola.

SUMMARY OF THE INVENTION

Taking the above into consideration, an object of the present inventionis to provide an apparatus for converting between different videoformats to enable visual communications between users of incompatibletelevision systems, which maintains the smoothness of object motions bypreventing unnatural effects from being introduced in the process ofvideo format conversion.

Further, another object of the present invention is to provide a videoformat conversion method for visual communications between users ofincompatible television systems, which maintains the smoothness ofobject motions by preventing unnatural effects from being introduced inthe process of video format conversion.

To accomplish the first object, according to the present invention,there is provided an apparatus for converting from a first video signalin a first video format to a second video signal in a second videoformat that is incompatible with the first video format. This apparatuscomprises: a frame interpolator which produces interpolated frames fromthe first video signal by using motion vectors obtained therefrom; and avideo signal generator which produces the second video signal from theinterpolated frames.

To accomplish the second object, according to the present invention,there is provided a method of converting from a first video signal in afirst video format to a second video signal in a second video formatthat is incompatible with the first video format. This method comprisesthe steps of: producing interpolated frames from the first video signalby using motion vectors obtained therefrom; and producing the secondvideo signal from the interpolated frames.

The above and other objects, features and advantages of the presentinvention will become apparent from the following description when takenin conjunction with the accompanying drawings which illustrate preferredembodiments of the present invention by way of example.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual view of a video format converter according to thepresent invention;

FIG. 2 is a block diagram of a video codec;

FIG. 3 is a block diagram of a frame interpolator according to a firstembodiment of the present invention;

FIG. 4 is a diagram showing the concept of forward-reference andbackward-reference motion vectors;

FIG. 5 is a block diagram of a video signal generator of the firstembodiment;

FIG. 6 is a diagram which shows inputs and outputs of frame memories;

FIGS. 7 and 8 are diagrams showing frame interpolation processes in thefirst embodiment;

FIG. 9 is a diagram showing the relationships between two video streamsin the first embodiment;

FIG. 10 is a block diagram of a frame interpolator according to a secondembodiment of the present invention;

FIG. 11 is a block diagram of a video signal generator of the secondembodiment;

FIG. 12 is a diagram which shows inputs and outputs of frame memories;

FIG. 13 is a diagram showing such a situation where the proposedconverter encounters a scene change when producing interpolated framesin normal mode;

FIGS; 14 and 15 are diagrams showing frame interpolation processes inthe second embodiment;

FIG. 16 is a diagram showing the relationships between two video streamsin the second embodiment;

FIG. 17 is a flowchart showing a process of video format conversionaccording to the present invention;

FIG. 18 is a table showing some key specifications of NTSC, PAL, andSECAM;

FIG. 19 is a table showing some key specifications of CIF and QCIF; and

FIG. 20 is a diagram showing a simplified system structure forinternational digital video communications between two countries usingdifferent television system standards.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described belowwith reference to the accompanying drawings.

FIG. 1 shows the concept of a video format converter according to thepresent invention. This proposed video format converter 1, comprising aframe interpolator 10 and a video signal generator 20, performs videoformat conversion to send visual information to a remote end that uses adifferent television system. Here, the term “video format conversion”refers to a process to convert frame frequencies.

According to the present invention, the frame interpolator 10 producesinterpolated frames from a first video signal given in a first videoformat, by using motion vectors obtained therefrom. (Technical detailsof frame interpolation will be described later.) From the interpolatedframes, the video signal generator 20 produces a second video signal ina second video format that is different from the first video format.

Recall that conventional video format converters use simple framesubsampling or frame insertion techniques to convert from one framefrequency to the other. In contrast to this, the proposed video formatconverter 1 uses frame interpolation techniques to obtain a new framefrequency. That is, the video format converter 1 is designed to generateinterpolated frames from the first video signal, and with theseinterpolated frames, produce the second video signal having a differentframe frequency. It should be noted here that the proposed converter 1produces interpolated frames in an adaptive manner, taking intoconsideration the difference between consecutive frames, which isrepresented by motion vectors. Because of the use of interpolated framesto recreate frames of the second video signal, the video formatconverter 1 of the present invention provides smooth motion in a scene,without awkward frame drops or other unnatural effects.

A more specific implementation of the above-described video formatconverter 1 will be described below. FIG. 2 shows a structure of a videocodec 100 which processes video signals by using internationallystandardized video processing algorithms. This video codec 100 acceptsvideo signals from two cameras, camera-A and camera-B, both having aframe frequency f1, while its internal common video format has anotherframe frequency f2. An external video monitor requires incoming videosignals to have still another frame frequency f3.

A video signal switching unit 110 is interposed between the cameras andvideo signal switching unit 110 to select either of the two videosignals supplied from the camera-A and camera-B. The selected videosignal having the first frame frequency f1 is applied to the video codec100. Switching from camera-A to camera-B, or vice versa, will cause aninstant and entire transition of screen images, which is referred to asa “scene change.” The video signal switching unit 110 produces a scenechange indication signal Ds to inform the video codec 100 of theoccurrence of such a scene change.

As the name implies, the video codec 100 provides two major functions,video coding and decoding, employing two dedicated internal data pathsfor them. Referring to its functional blocks shown in FIG. 2, one datapath running from left to right serves as a video coder, while the otherdata path running from right to left serves as a video decoder. Thefollowing section will explain the elements of each data path, as wellas their operation.

The video coding path begins with a video format converter 1-1, whichcomprises a frame interpolator 10-1 and a video signal generator 20-1.The frame interpolator 10-1 produces interpolated frames from a sourcevideo signal supplied from the video signal switching unit 110, by usingmotion vectors obtained from consecutive frames of the source videosignal. With the interpolated frames produced, a video signal generator20-1 produces another video signal stream having a frame frequency f2.When a scene change occurs, the video signal generator 20-1 receives thescene change indication signal Ds from the video signal switching unit110, and based on this signal, it selects appropriate interpolatedframes that are derived from correlated source frames, to continuouslyproduce the converted video signal.

The video signal produced by the video format converter 1-1 is thensupplied to a coder 101 a to perform a data compression, or reduce itsdata size. More specifically, the coder 101 a performs digital cosinetransform (DCT) coding, quantization, and motion compensation, tocompress spatial and temporal redundancies of video frames. After that,it applies variable-length coding to produce outgoing data coded in aprescribed format. The actual amount of the produced data may vary frameby frame, and hence the data rate. A transmission buffer 102 a serves astemporary storage to smooth out such variations of the data rate. Thebuffer occupancy ratio is fed back to a coding controller 103, whichcontrols the amount of coded data being produced by the coder 110 a,thus regulating the data flow to the transmission buffer 102 a. At thefinal stage of the video coding path, a transmission coder 104 aprovides several functions necessary for data transmission. Forinstance, it inserts dummy bits when the transmission buffer 102 a isempty, and it adds error correction code to the coded bit stream to betransmitted.

The video decoding path, on the other hand, begins with a transmissiondecoder 104 b which receives a coded bit stream from a remote end. Itremoves dummy bits from the received bit stream, as well as performserror check and correction. A reception buffer 102 b regulates the flowof incoming data to ensure that each frame be decoded within aprescribed period. The received bit stream is then supplied to a decoder101 b, where a variable-length decoding algorithm is applied so as toextract each piece of coded data from the bit stream. The decoder 101 bfurther executes dequantization, inverse-DCT, and motion-compensatedimage reconstruction, thereby expanding the coded data.

The video signal reconstructed by the decoder 101 b has a framefrequency f2, which is not compatible with the external video monitor.For frequency conversion, the video signal is supplied to a video formatconverter 1-2, which comprises a frame interpolator 10-2 and a videosignal generator 20-2. The frame interpolator 10-2 produces interpolatedframes from the supplied video signal by using motion vectors. Fromthese interpolated frames, the video signal generator 20-2 produces anew sequence of frames, or a video signal with a different framefrequency, f3, which is compatible with the video monitor.

The format of incoming coded bit stream may change from CIF to QCIF, orQCIF to CIF, during operation. When such a common format switchover isencountered, the video signal generator 20-2 selects appropriateinterpolated frames that are derived from correlated source frames, tocontinue to produce the video signal with the frame frequency f3.

A first embodiment of the present invention will now be described belowwith reference to FIGS. 3 to 9. In this first embodiment, a video formatconverter reduces the frame frequency. For illustrative purposes, it isassumed here that the proposed converter is designed to make a framefrequency conversion from 30 Hz to 25 Hz.

Referring to a block diagram of FIG. 3, a frame interpolator 10 adesigned for video format conversion to a lower frame frequency isprovided. This frame interpolator 10 a produces three kinds ofinterpolated frames FLa, FLb, and FLc from a first video signal. Itemploys four frame memories M1 to M4, which are connected in series tostore four past frames of the first video signal being supplied from anexternal source. As FIG. 3 shows, the symbol for the present incomingframe is “M1in,” while those for the outputs of the frame memories M1,M2, M3, and M4 are “M2out,” “M3out,” “M4out,” and “M5out,” respectively.

A motion detector MC produces forward-reference motion vectors by makinga comparison between the frames M2out and M3out. It also producesbackward-reference motion vectors by making a comparison between theframes M1in and M2out. FIG. 4 explains the concept of forward-referenceand backward-reference motion vectors. Suppose here that the frame M2outis now being subjected to the motion detection process (i.e., M2out isthe current picture). When observed from this frame M2out, the frameM3out is regarded as its previous frame, and the frame M1in as itsfuture frame. A forward-reference motion vector Vf represents thedirection and distance of a moving object observed between the framesM3out and M2out. On the other hand, a backward-reference motion vectorVa represents those observed between the frames M2out and M1in.

Referring again to FIG. 3, the frame interpolator 10 a has two delayunits D1 and D2 each of which provides a one-frame delay to givenforward-reference and backward-reference motion vectors Vf and Va. Thatis, forward-reference and backward-reference motion vectors Vf and Vaappearing at the output of the delay unit D1 have a delay of one frameinterval, with respect to those at the output of the motion detector MC,while those appearing at the output of the delay unit D2 have a delay oftwo frame intervals.

The above-noted motion vector outputs of the motion detector MC anddelay units D1 and D2 are supplied respectively to three multipliers 11a, 11 b, and 11 c for multiplication of vector weighting coefficientsC1, C2, and C3. Here, the vector weighting coefficients C1 to C3 denotethe mixture ratios of forward-reference motion vectors andbackward-reference motion vectors. By multiplying such coefficients,each multiplier 11 a to 11 c calculates a weighted mixture offorward-reference motion vectors Vf and backward-reference motionvectors Va thereby outputting weighted motion vectors MV1 to MV3.According to those weighted motion vectors MV1 to MV3, variable delayunits VD1 to VD3 apply motion compensation to the frames M3out to M5out,respectively. They output the resultant frames FL1 to FL3 in phase withthe three frames M3out to M5out, respectively.

Switches SW1 to SW3 select video frames according to a motion/stillselection command provided from an external source. More specifically,they select the outputs of the variable delay units VD1 to VD3 when themotion/still selection command indicates motion picture mode. In turn,they select the outputs of the frame memories M2 to M4 when the commandindicates still picture mode. Picture elements of the selected framesare then supplied to multipliers 12 a to 12 c for multiplication offrame weighting coefficients k1 to k3. Here, the coefficients k1 to k3give a mixture ratio of the three selected frames. The resultantweighted frame signals FL1-m to FL3-m are supplied to adders at the nextstage for frame summation.

There are three adders 13 a to 13 c to obtain three interpolated frames.The first adder 13 a produces an interpolated frame FLa by adding thelast two weighted frames FL2-m and FL3-m. The second adder 13 b producesanother interpolated frame FLb by adding all the three weighted framesFL1-m, FL2-m and FL3-m. The third adder 13 c produces still anotherinterpolated frame FLc by adding the first two weighted frames FL1-m andFL2-m.

The next section will now explain a specific structure of a video signalgenerator designed for video format conversion to a lower framefrequency (e.g., 30 Hz to 25 Hz). FIG. 5 is a block diagram of a videosignal generator 20 a. This video signal generator 20 a comprises aswitch SW21 a, a FIFO 22 a, a FIFO controller 24 a, and a frameswitching controller 27.

The frame switching controller 27 comprises difference detectors 27 a to27 d and a switching controller 27 e. The difference detector 27 adetects differences between the frames M1in and M2out. The differencedetector 27 b detects differences between the frames M2out and M3out.The difference detector 27 c detects differences between the framesM3out and M4out. The difference detector 27 d detects differencesbetween the frames M4out and M5out. Examining the detected differences,the switching controller 27 e finds the maximum difference value. Thisinformation permits the switching controller 27 e to determine at whichframe position the scene change or common format switchover hasoccurred, based on a scene change indication signal Ds or a commonformat switching command Df supplied from external sources. Theswitching controller 27 e thus outputs an appropriate switching controlcommand SWC to the switch SW21 a.

The switch SW21 a has three input terminals “a,” “b,” and “c” to receiveinterpolated frames FLa to FLc, the outputs of the adders 13 a to 13 cdescribed in FIG. 3. The switch SW21 a selects either one of the threeinputs according to the switching control command SWC as will bedescribed later.

The FIFO 22 a serves as a buffer memory for the interpolated framesbeing supplied from the switch SW21 a at the rate of 30 Hz. It controlstheir output timings under the control of the FIFO controller 24 a, sothat interpolated frames will be sent out at 25 Hz.

The switch SW21 a is controlled as follows. The switch SW21 a has twomodes of operation: normal mode and transitional mode. In normal mode,the switch SW21 a keeps its contact sw-1 a at the second position “b,”and thus the FIFO 22 a receives interpolated frames FLb at a rate of 30Hz. The FIFO 22 a decimates some of the received frames at predeterminedintervals, thereby converting the frame frequency down to 25 Hz.

Transitional mode, as opposed to the normal mode, is selected at scenechanges, as well as in the case that the common video format is switchedfrom CIF to QCIF or vise versa (i.e., common format switchover). Whilelater explanation assumes the occurrence of a scene change, thoseskilled in the art will appreciate that the same will apply to theoperation in the case of common format switchover.

FIG. 6 shows inputs and outputs of the frame memories M1 to M4 (FIG. 3).The symbol “M1in” is used to represent a source frame being entered tothe first frame memory M1, while “M2out” to “M5out” represent the outputpictures of the frame memories M1 to M4, respectively. Those fiveconsecutive pictures represent a part of a given frame sequence. FIG. 6shows three instances of such frame snapshots taken at times t0, t1, andt2, and in this progression, the pictures are shifted by one frametoward the right-hand side of FIG. 6.

This example video sequence of FIG. 6 includes a scene change, whichactually happened at some time before t0. The switching controller 27 eis designed to initiate transitional mode operation when such a scenechange is observed between the frames M3out and M4out. Here, theswitching control command SWC serves as a message to inform the switchSW21 a that a scene change has reached that critical point. This messagemakes the switch SW21 a enter the transitional mode and handle the scenechange at times t0 and t1. That is, the transitional mode operationlasts only two frame intervals, and the switch SW21 a returns to normalmode at time t2 to resume normal switching operations.

At time t0 (i.e., the first half of transitional mode operation), theswitch SW21 a first has to confine the source of interpolated frames toframes captured before the occurrence of the scene change. To this end,the switch SW21 a moves its contact sw-1 a to the first position “a”according to the switching control command SWC, thereby selecting aninterpolated frame FLa derived from the frames M4out and M5out (i.e.,old scene). After that, at time t1, the switch SW21 a moves the contactsw-1 a to the third position “c” so as to select interpolated framesderived only from a new scene. Accordingly, an interpolated frame FLcproduced from the frames M3out and M4out is selected. At time t2, theswitch SW21 a resumes its normal mode operation by returning its contactsw-1 a to the second position “b,” again selecting interpolated framesFLb being produced from M3out, M4out, and M5out.

The switch SW21 a is designed to change the position of its contact sw-1a step by step (i.e., “b”-“c”-“a”-“b”) according to each switchingcontrol command SWC. However, it is not intended that the invention belimited to this structure, but the switch SW21 a can also be configuredto do the same autonomously at predetermined intervals, once it receivesthe first switching control command SWC.

The next section will now describe a process of producing interpolatedframes for video format conversion to a lower frame frequency (e.g., 30Hz to 25 Hz).

FIGS. 7 and 8 present a frame interpolation process including no scenechange or common format switchover. The symbol “Fin” represents a sourceframe sequence having a frame frequency of 30 Hz that is given to theframe interpolator 10 a. For the sake of convenience, source frames arelabeled “F1” to “F6” in a cyclic manner. Consider that the firstinstance of “F1” appears at the output of the fourth frame memory M4.Then the frames F2, F3, and F4 are at the outputs of the frame memoriesM3, M2, and M1, respectively, and F5 at the input of the first framememory M1. At the next cycle, the frame F2 appears at the output of theframe memory M4, since the frame memory contents are shifted forward byone frame. Likewise, the frames F3, F4, and F5 are now at the outputs ofthe frame memories M3, M2, and M1, respectively, and F6 at the input ofthe first frame memory M1.

On the other hand, the symbol “Fout” shows a series of interpolatedframes FLb produced by the video signal generator 20 a when the switchSW21 a is in normal mode (i.e., its contact is at the second position“b”). The interpolated frames Fout are labeled “F1 a” to “F5 a” in acyclic manner

In the process shown in FIGS. 7 and 8, interpolated frames are producedfrom the following combinations of source frames:

F1 a produced from F1, F2, and F3

F2 a produced from F2, F3, and F4

F3 a produced from F3, F4, and F5

F4 a produced from F5, F6, and F1

F5 a produced from F6, F1, and F2

Note here that, although the frame interpolator 10 a producesinterpolated frames from frames F4 to F6, the FIFO 22 a in the videosignal generator 20 a will never output them. That is, the proposedvideo format converter subsamples the produced interpolated frames toreduce its frame frequency.

FIG. 9 depicts the above-noted relationships between source frames (30Hz) and subsampled frames (25 Hz). Each bold solid line represents aframe, while broken lines indicate unit time intervals. The interpolatedframes Fout shown in FIGS. 7 and 8 are fed to the FIFO 22 a at the rateof 30 Hz. The FIFO 22 a regulates the intervals of output frames so thatthey will be sent out at 25 Hz.

A second embodiment of the present invention will now be describedbelow, with reference to FIGS. 10 to 17. As opposed to the firstembodiment, a video format converter proposed of the second embodimentis designed to raise the frame frequency. For illustrative purposes, itis assumed here that the proposed converter makes a frame frequencyconversion from 25 Hz to 30 Hz.

Referring first to a block diagram of FIG. 10, a frame interpolator 10 bdesigned to raise the frame frequency will be described below. Thisframe interpolator 10 b produces four kinds of interpolated frames FLa,FLb, FLc, and FLd from a first video signal. This frame interpolator 10b differs from the frame interpolator 10 a of FIG. 3 in that a switchSW4 and an adder 13 d are newly employed. Since the other functionalelements of the frame interpolator 10 b are common to those of the frameinterpolator 10 a, the following explanation will focus on theirdifferences, while maintaining like reference numerals to like elements.

The adder 13 d produces a fourth interpolated frame FLd by adding theweighted frames FL1-m and FL2-m. The switch SW4 is controlled through aswitching control command SWCa. While being normally closed, this switchSW4 is turned off when a scene change or common format switchover isdetected, in which case the interpolated frame FLd is identical to theframe FL1-m, since the adder 13 d becomes transparent. More details ofthe switch SW4 and switching control command SWCa will be providedlater.

Referring next to FIG. 11, a video signal generator 20 b designed toraise the frame frequency will be described below. This video signalgenerator 20 b comprises two switches SW21 b and SW22 b, two FIFOs 22 band 25 b, a FIFO controller 24 b, a first frame switching controller27-1, and a second frame switching controller 26 b.

The first frame switching controller 27-1 has difference detectors 27 ato 27 d and a switching controller 27 e-1. The difference detector 27 adetects differences between the frames M1in and M2out. The differencedetector 27 b detects differences between the frames M2out and M3out.The difference detector 27 c detects differences between the framesM3out and M4out. The difference detector 27 d detects differencesbetween the frames M4out and M5out. By examining the detecteddifferences, the switching controller 27 e-1 finds the maximumdifference value. This information permits the switching controller 27e-1 to determine at which frame position the scene change or commonformat switchover has occurred, based on a scene change indicationsignal Ds or a common format switching command Df supplied from externalsources. The switching controller 27 e-1 thus outputs an appropriateswitching control command SWC to control the first switch SW21 b. Itfurther produces another switching control command SWCa to control theswitch SW4 in the frame interpolator 10 b.

The first switch SW21 b has three input terminals “a,” “b,” and “c” toreceive three kinds of interpolated frames FLa, FLb, and FLc, theoutputs of the adders 13 a to 13 c shown in FIG. 10. This first switchSW21 b selects either one of the three inputs according to the switchingcontrol command SWC, and sends the selected frames to the first FIFO 22b. On the other hand, the second FIFO 25 b receives another kind ofinterpolated frames, FLd, from the frame interpolator 10 b. Note thatthose interpolated frames are generated at 30 Hz, i.e., the rate ofsource video frames. The FIFO controller 24 b supplies control signalsto the two FIFOs 22 b and 25 b to regulate the flow of outgoing frames,so that they will be output at 25 Hz.

The outputs of the two FIFOs 22 b and 25 b are then supplied to twoinput terminals “d” and “e” of the second switch SW22 b, respectively.According to frame switching commands Dc from the controller 26 b, thesecond switch SW21 b chooses one of the two input signals as finaloutput of the video signal generator 20 b.

Referring to FIGS. 10 to 12, the following section will describe how theswitches SW4, SW21 b, and SW22 b operate in the second embodiment. As inthe first embodiment, those switches have two modes of operation: normalmode and transitional mode. The latter mode is selected in the case ofscene changes or common format switchover. Although the case of a scenechange will be discussed later, those skilled in the art will appreciatethat the same will apply to common format switchover.

FIG. 12 shows inputs and outputs of the frame memories M1 to M4 (FIG.10). The symbol M1in is used to represent a source frame being enteredto the first frame memory M1, while M2out to M5out represent the outputpictures of the frame memories M1 to M4, respectively. These fiveconsecutive pictures represent a part of a given frame sequence. FIG. 12shows four instances of such frame snapshots taken at times t0, t1, t2,and t3, and in this progression, the pictures are shifted by one frame,toward the right-hand side of FIG. 12.

In normal mode, the SW4 is closed and the first switch SW21 b keeps itscontact sw-1 b at the second position “b.” On the other hand, the secondswitch SW22 b connects its contact sw-2 b at the first position “d”according to the frame switching command Dc, which allows the producedframes to be transmitted through the First FIFO 22 b and second switchSW21 b in the following sequence:

FLb-1 produced from frames M3out to M5out at time t0

FLb-2 produced from frames M3out to M5out at time t1

FLb-3 produced from frames M3out to M5out at time t2

FLb-4 produced from frames M3out to M5out at time t3

When the interpolated frame FLb-3 is output, the adder 13 d producesanother interpolated frame FLd from the frames M3out and M4out andsupplies it to the second FIFO 25 b. After the interpolated frame FLb-4is sent out, the frame switching controller 26 b alters its commandoutput Dc to make the first FIFO 22 b move its contact sw-2 b to theposition “e” so as to select the frame FLd held in the second FIFO 25 b.The frame switching controller 26 b then commands the second switch SW21b to return its contact sw-2 b to the position “d,” thus repeating theabove-described operation.

If a scene change is encountered, the proposed video format converterenters to transitional mode. The operation in this mode will bedescribed below with reference to FIGS. 6, 10, and 11.

As described earlier, the example video sequence of FIG. 6 includes ascene change. The switching controller 27 e-1 (FIG. 11) is designed toinitiate transitional mode operation, when such a scene change isobserved between the frames M3out and M4out. Here, the switching controlcommand SWC serves as a message to inform the first switch SW21 b thatthe scene change has reached that critical point. This message makes thefirst switch SW21 b enter the transitional mode and handle the scenechange at times t0 and t1. The first switch SW21 b then returns tonormal mode at time t2 to resume normal switching operations.

At time t0, the first switch SW21 b has to confine the source ofinterpolated frames to frames captured before the occurrence of thescene change. To this end, the first switch SW21 b moves its contactsw-1 b to the first position “a” according to the switching controlcommand SWC, thereby selecting an interpolated frame FLa produced fromthe frames M4out and M5out. At the same time, the second switch SW22 bsets its contact sw-2 b to the first position “d” according to the frameswitching command Dc. As a result, the interpolated frame FLa is sentout.

At time t1, the first switch SW21 b has to select an interpolated frameproduced only from the new scene. The first switch SW21 b thus moves itscontact sw-1 b to the third position “c,” thereby selecting aninterpolated frame FLc produced from the frames M3out and M4out. Thesecond switch SW22 b maintains its contact position “d” according to theframe switching command Dc, thus sending out the selected interpolatedframe FLc.

At time t2, the first switch SW21 b returns its contact sw-1 b to thesecond position “b” to resume normal mode operation, again selectinginterpolated frames FLb. The second switch SW22 b keeps its contactposition “d” according to the frame switching command Dc, thus sendingout the selected interpolated frame FLb.

As discussed above, the first switch SW21 b is designed to change theposition of its contact sw-1 b step by step (i.e., “b”-“c”-“a”-“b”)according to each switching control command SWC. However, it is notintended that the invention be limited to this structure, but the firstswitch SW21 b can also be configured to do the same autonomously atpredetermined intervals, once it receives the first switching controlcommand SWC.

FIG. 13 shows such a situation where the proposed converter encounters ascene change when producing an interpolated frame FLd in normal mode.Although it is normally closed, the switch SW4 is designed to becomeopen when a scene change is observed at a particular point shown in FIG.13. This means that the aforementioned switching control command SWCaserves as a message to inform the switch SW4 that a scene change hasreached that critical point. With the switch SW4 turned off, the adder13 d produces an interpolated frame FLd solely from the frame M3out.

Referring now to FIGS. 14 and 15, the next section will describe aprocess of producing interpolated frames to convert the video format toa higher frame frequency (e.g., 25 Hz to 30 Hz). This illustratedprocess, however, includes no scene change or common format switchover.

The symbol “Fin” represents a source frame sequence having a framefrequency of 25 Hz that is given to the frame interpolator 10 b. For thesake of convenience, source frames are labeled “F1” to “F5” in a cyclicmanner. Consider that the first instance of “F1” appears at the outputof the fourth frame memory M4. Then the frames F2, F3, and F4 are at theoutputs of the frame memories M3, M2, and M1, respectively, and F5 atthe input of the first frame memory M1.

At the next cycle, the frame F2 appears at the output of the framememory M4, since the frame memory contents are shifted forward by oneframe. Likewise, other frames F3, F4, and F5 are now at the outputs ofthe frame memories M3, M2, and M1, respectively, and the second instanceof F1 appears at the input of the first frame memory M1.

On the other hand, the symbol “Fout1” shows a series of interpolatedframes FLb produced by the video signal generator 20 b when the firstswitch SW21 b is in normal mode (i.e., its contact is at the secondposition “b”). Those frames Fout1 are labeled “F1 a,” “F2 a,” “F3 a,”“F5 a,” and “F6 a” in a cyclic manner. Further, the symbol “Fout2” showsadditional interpolated frames FLd produced by the adder 13 d in theframe interpolator 10 b. In FIGS. 14 and 15, such interpolated framesare labeled “F4 a.”

The above process as a whole generates six interpolated frames in thefollowing sequence during five source frame intervals:

F1 a produced from F1, F2, and F3

F2 a produced from F2, F3, and F4

F3 a produced from F3, F4, and F5

F4 a produced from F4 and F5

F5 a produced from F4, F5, and F1

F6 a produced from F5, F1, and F2

Note here that the frames F3 a and F4 a are produced at the same time.The flow of produced frames is regulated by controlling the FIFOs, thusobtaining final output at a frame frequency of 30 Hz.

FIG. 16 depicts the relationships between source frame sequence (25 Hz)and converted frame sequence (30 Hz). Each bold solid line represents aframe, while broken lines indicate unit time intervals. The interpolatedframes Fout1 and Fout2 shown in FIGS. 14 and 15 are fed to the FIFOs 22b and 25 b for output flow control. The FIFOs 22 b and 25 b serve asbuffer storage to regulate the intervals of output frames so that theywill be sent out at 30 Hz.

As described above, the video format converter 1 of the presentinvention uses frame interpolation techniques to obtain a new framefrequency. That is, the video format converter 1 is designed toadaptively generate interpolated frames from the first video signal byconsidering motion vectors obtained therefrom. A second video signal isproduced from the interpolated frames, so that it will have a differentframe frequency. Accordingly, the video format converter 1 of thepresent invention provides smooth motion in a scene, eliminatingdiscontinuity in a frame sequence, which could be introduced in theprocess of frame frequency conversion.

The following section will now describe a video format conversion methodaccording to the present invention. FIG. 17 is a flowchart showing aprocess to convert from a first video signal in a first video format toa second video signal in a second video format that is incompatible withthe first video format. The process comprises the following steps:

(S1) Interpolated frames are produced from the first video signal byusing motion vectors of the first video signal.

(S2) The second video signal is produced from the interpolated frames.

More specifically, the difference between the two video formats lies intheir frame frequencies. That is, the proposed method performs frameinterpolation of the first video signal having a first frame rate,thereby generating the second video signal having a second framefrequency.

The above-described embodiments of the present invention is nowsummarized as follows. According to the present invention, a method andapparatus for converting between different video formats are proposed.They produce interpolated frames from a first video signal, and fromthese interpolated frames, generate a second video signal that isincompatible with the first video signal. With appropriate frameinterpolation techniques, the present invention promises smooth videomotion in visual communications between remote users of differenttelevision systems, eliminating unnatural effects which could beintroduced in the process of video format conversion.

The foregoing is considered as illustrative only of the principles ofthe present invention. Further, since numerous modifications and changeswill readily occur to those skilled in the art, it is not desired tolimit the invention to the exact construction and applications shown anddescribed, and accordingly, all suitable modifications and equivalentsmay be regarded as falling within the scope of the invention in theappended claims and their equivalents.

What is claimed is:
 1. An apparatus to convert a first video signal in afirst video format to a second video signal in a second video formatthat is incompatible with the first video format, comprising: a frameinterpolator that produces a plurality of interpolated frames from thefirst video signal by using motion vectors obtained therefrom, theplurality of interpolated frames being derived from differentcombinations of consecutive frames of the first video signal; a switchthat selects one of the plurality of interpolated frames to produce thesecond video signal; a plurality of difference detectors that detectdifferences between the consecutive frames of the first video signals;and a switching controller that finds a change in the first video signalby examining the differences detected by the difference detectors andcontrols the switch to choose such interpolated frames that are derivedonly from correlated source frames before the change or after the changein the first video signal.
 2. The apparatus according to claim 1,wherein the frame interpolator produces a plurality of frames by usingthe motion vectors for motion compensation, multiplies the producedframes by weighting coefficients, obtains weighted sums by adding theweighted frames, and outputs the weighted sums as the interpolatedframes.
 3. The apparatus according to claim 1, wherein the frameinterpolator uses forward-reference motion vectors andbackward-reference motion vectors as the motion vectors.
 4. Theapparatus according to claim 1, wherein the frame interpolator producesthe interpolated frames, while switching between still picture mode andmotion picture mode.
 5. The apparatus according to claim 1, wherein: thefirst video format has a first frame frequency; and the second videoformat has a second frame frequency that is different from the firstframe frequency.
 6. The apparatus according to claim 1, wherein thechange in the first video signal includes a scene change.
 7. Theapparatus according to claim 1, wherein the change in the first videosignal includes a common format switchover.
 8. A method for converting afirst video signal in a first video format to a second video signal in asecond video format that is incompatible with the first video format,comprising: producing a plurality of interpolated frames from the firstvideo signal by using motion vectors obtained therefrom, the pluralityof interpolated frames being derived form different combinations ofconsecutive frames of the first video signal; detecting differencesbetween the consecutive frames of the first video signal; and finding achange in the first video signal by examining to differences detected;when no change is detected, selecting one of the interpolated frames toproduce the second video signal; and when the change is detected,selecting one of the interpolated frames that are derived only from thecorrelated source frames before the change or after the change in thefirst video signal.
 9. The method according to claim 8, wherein: thefirst video format has a first frame frequency; and the second videoformat has a second frame frequency that is different from the firstframe frequency.
 10. The method of claim 8, wherein the change in thefirst video signal includes a scene change.
 11. The method of claim 8,wherein the change in the first video signal includes a common formatswitchover.